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We have searched for second generation leptoquark (LQ) pairs in the /i/i+jets channel using 
94 ± 5 pb^^ of pp cohider data collected by the D0 experiment at the Fermilab Tevatron during 
1993-1996. No evidence for a signal is observed. These results are combined with those from the 
/ii^+jets and w-\-]ets channels to obtain 95% confidence level (C.L.) upper limits on the LQ pair 
production cross section as a function of mass and 13, the branching fraction of a LQ decay into a 
charged lepton and a quark. Lower limits of 200(180) GeV/c^ for (3 — 1(|) are set at the 95% C.L. 
on the mass of scalar LQ. Mass limits are also set on vector leptoquarks as a function of j3. 



The observed symmetry in the spectrum of fundamen- 
tal particles between leptons (l) and quarks (q) has led 
to suggestions of the existence of leptoquarks (LQ) [Q. 
Leptoquarks would carry both lepton and quark quan- 
tum numbers, and would decay to Iq systems. Although, 
in principle, leptoquarks could decay to any Iq combi- 
nations, limits on flavor-changing neutral currents, rare 
lepton-family violating decays, and proton decay, sug- 
gest that leptoquarks would couple only within a single 
generation M. This implies the existence of three LQ 
generations, analogous to the fermion generations in the 
standard model. 

At the Fermilab Tevatron, leptoquarks are predicted 
1^ to be produced dominantly via gluon {g) splitting, 
pp —^ g + X —^ LQLQ + X. This Letter reports on an 
enhanced search for second generation Icptoquark pairs 
produced in pp interactions at a center-of-mass energy 
y/s = L8 TeV. The experimental signature considered 
is when both leptoquarks decay via LQ —^ fj,q, where q 
can be either a strange or a charm quark depending on 
the electric charge of the LQ. The corresponding experi- 
mental cross section is /3^ x a{pp -^ LQLQ), where (3 is 
the unknown branching fraction of a LQ to a muon (/i) 
and a quark (jet). 

Previous studies by the D0 [|| and CDF § collabo- 
rations have considered pair production of scalar lepto- 
quarks in ^/^+jcts final states. These studies provide 
lower limits on the mass of LQs of 119 GeV/c^ and 
202 GeV/c^, respectively, for (3 = 1. Lower limits of 
160 GeV/c^ for /? = 1/2 were obtained by D0 from the 
/ij/-|-jets final state |6| and by CDF from the /i/i-|-jets fi- 
nal state P]. For /3 = 0, D0 has obtained a lower limit 
of 79 GeV/c^ from the i^t^-fjets channel M. 

The present study is complementary to previous D0 
searches in the iiv+iets H] and i^z^-fjets final states, 
and greatly extends the previous search in the /i/i+jets 
channel Q . The sensitivity for detection of leptoquarks 
is increased by considering a larger data set that uses 
the calorimeters to identify muon candidates, and em- 
ploys several optimization techniques to enhance effi- 
ciency. These results are combined with results from 
other decay channels to improve mass limits on LQs. (A 
detailed description of this analysis can be found in Ref. 

The D0 detector [g[ consists of three major compo- 
nents: an inner detector for tracking charged particles, a 
uranium/liquid argon calorimeter for measuring electro- 
magnetic and hadronic showers, and a muon spectrome- 
ter consisting of magnetized iron toroids and three lay- 
ers of drift tubes. Jets are measured with an energy 
resolution of approximately a{E)/E = O.S/a/B [E in 
GeV) . Muons are measured with a momentum resolution 
of a{l/p) = 0.18(]5 - 2)/p2 © 0.003 [p in GcV/c). 

Event samples are obtained from triggers requir- 
ing the presence of a muon candidate with trans- 
verse momentum p^ > 5 GeV/c in the fiducial region 



|ry^| < 1.7 [t] = — ln[tan(i0)], where 9 is the polar angle 
of a track with respect to the z-axis taken along the 
direction of the proton beam), and at least one jet can- 
didate with transverse energy Eip > 8 GeV and |?7j| < 
2.5. The data correspond to an integrated luminosity of 
94 ± 5 pb-i collected during the 1993-1995 and 1996 
Tevatron collider runs at Fermilab |10| . 

Jets are measured in the calorimeters and are re- 
constructed offline with a cone algorithm having ra- 
dius TZ = ^/AxtP + Arf = 0.5. In the final event sample, 
two or more jets are required with E^ > 20 GeV within 
|77,|<3.0. 

Muon candidates reconstructed in the muon spectrom- 
eter are required to have a track that projects back to 
the interaction vertex. The track is required to be con- 
sistent with a muon of p^ > 20 GeV/c and jry^l < 1.7. 
In addition, the muon is required to deposit energy in 
the calorimeter consistent with the passage of a min- 
imum ionizing particle (MIP). To reduce backgrounds 
from heavy quark production, candidate muons are re- 
quired to be isolated from all jets passing the selection 
criteria listed above by A_R^j > 0.5 in the r] — (j) plane. 

Single muon candidates can also be tracked in the 
calorimeters, where an isolated high-py muon deposits 
only a small fraction of its total energy. This re- 
sults in a unique energy signature consisting of energy 
from a MIP (ii^Mip) ^j^ and a large transverse en- 
ergy imbalance (^y) in the calorimeter that is propor- 
tional to the muon momentum, and points in the az- 
imuthal direction of the E'mip ■ Muon candidates are re- 
stricted to the region \ri\ < 1.7, and are required to have 
\A(J>{Emip-$t)\ < 0-25 radians. The kinematic quanti- 
ties (e.g., pi^) of these candidates are calculated using 
the (77, (j)) direction of the Emip and the component of 
the $!rp along the azimuthal direction of the Emip- 

Dimuon candidate events are required to have two 
muons with pl^ > 20 GeV/c. At least one muon must 
be in the central muon spectrometer (|77^| < 1.0). A sec- 
ond nuion with jry^j < 1.7 may be identified using either 
the muon spectrometer or the calorimeters. 

After obtaining a sample of /u/z-l-jets events, a selec- 
tion is applied to the event topology. Heavy LQ pairs 
are expected to have a smaller Lorentz boost, and to de- 
cay more symmetrically, than the background events. To 
take advantage of these differences, the sphericity in the 
center-of-mass frame (Scm) is required to be greater than 
0.05. ScM is defined as 1.5(Ai -|- A2), with Ai < A2 < A3 
being the normalized eigenvalues of the momentum ten- 
sor. The momentum tensor is formed from the Et (pt) 
of all jets (muons) in an event, and 5cm = (1) corre- 
sponds to a linear (spherical) topology. 

Leptoquark events are simulated with the ISAJET |1J] 
Monte Carlo event generator for scalar LQ (S'lq), and 
with PYTHIA ||l3| for vector LQ (Vlq)- The detection ef- 
ficiencies for S'lq and Vlq of the same mass are found to 
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FIG. 1. Invariant mass of /i^+jets events. The mass is 
calculated from all muons and jets that pass the selection cri- 
teria. The hatched regions give the background estimation, 
the square points are the p/i+jets data, and the triangular 
points are the prediction for Slq from the Monte Carlo. Un- 
certainties on bins with no data points are obtained from the 
68% confidence interval. 

be consistent within the uncertainties. For massive vec- 
tor leptoquarks (my^Q > 200 GcV/c^), efficiencies are 
insensitive to differences between minimal vector (MV, 
Kg = 1, Ag = (l|]) and Yang-Mills (YM, kg ^ Xg = 
|l4| ) couplings to standard model bosons ||l^. Conse- 
quently, the 5lq Monte Carlo is used to represent the 
shapes of distributions for both S'lq and T^lq analyses. 

The leptoquark cross sections for 5lq are next-to- 
leading-order calculations (NLO) jl^] at a renormaliza- 
tion scale fi = msj^Q ■ The uncertainties are determined 
from variation of the renormalization/factorization scale 
from 2msLQ to ^rns^Q ■ Both types of Vlq cross sections 



are calculated to leading-order (LO) at /i = my^^ |14 |. 

The dominant backgrounds are due to VF-|-jets and 
Z-|-jets production, and are simulated using VECBOS ||l^] 
at the parton level and HERWIG |1^ for parton frag- 
mentation. Background due to WW production is sim- 
ulated with PYTHIA ||l^. Background from ti produc- 
tion is simulated using HERWIG with a top quark mass 
of 170 GeV/c^. All Monte Carlo samples are processed 
through a detector simulation program based on the 
GEANT ^ package. 

After initial selection, there are 53 events in the 
data sample consistent with an estimated background 
of 53±13 events. The distribution in invariant mass 
(rrievent) Calculated from all muons and jets passing the 
selection criteria is given in Fig. 1. The largest expected 
background is from W^-|-jets (43±13 events) where $rp 
from a neutrino is misidentified as a second muon when 
low-energy jets or calorimeter noise mimic the energy 
signature of a MIP. The other backgrounds are from 
Z+jets events (5.6±0.9), WW events (2.3±0.9, consis- 
tent with previous experimental limits at D0 ^^), and 
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FIG. 2. Output of the neural network. The network calcu- 
lates a value for each event based on the inputs (see text) and 
a set of internal values which are determined during network 
training on Slq and background Monte Carlo. 

tt events (2.1±0.6). The uncertainty in the background 
estimate is dominated by the statistical uncertainty of 
the T4^-|-jets Monte Carlo and the systematic uncertainty 
in the W^-|-jets production cross section. The estimate 
for the production of 200 GeV/c^ scalar leptoquarks 
that pass all of the previous selection requirements is 
3.7±0.4 events. All leptoquark production estimates are 
for 200 GeV/c^ S'lq , and use the NLO cross section at a 
scale ^ = 2msLQ • 

A neural network (NN) analysis |2^ is employed to 
separate any possible signal from background. The 
NN is trained using a mixture of T4^-|-jets, Z+^ets, and 
ti background Monte Carlo events, and an indepen- 
dently generated S'lq Monte Carlo sample for a mass 
TO5lq = 200 GeV/c^. The NN uses seven inputs 



njrp , p^" , p^' 



't^\pti?, (^^^ +E'^-), mevent and {E'^' +Ei^)/Y.Ei^, 
where jets (muons) are ordered in Et (pt)], and 15 nodes 
in a single hidden layer to calculate an output. The net- 
work output (Dnn) is shown in Fig. 2. 

No evidence of a signal is seen either in the Dnn dis- 
criminant or in any kinematic distribution. The Dnn 
selection is optimized for the calculation of limits using a 
measure of sensitivity |q] calculated from samples of Slq 
and background Monte Carlo. The requirement is set at 
Dnn > 0-9. For this selection no events are observed, 
consistent with an estimated background of 0.7±0.5 
events (0.49±0.16 ti, 0.15±0.04 Z-l-jets, 0.05±0.05 WW, 
and 0+!];g W+iets events). The estimate for 200 GeV/c^ 
Slq production is 3.3±0.3 events. 

The selection criteria are applied to the Monte Carlo 
for a range of LQ masses. The leptoquark detection ef- 
ficiencies, estimated to be 10%-26% depending on the 
LQ mass, are listed in Table I, along with the 95% confi- 
dence level (C.L.) upper limits on the cross sections. The 
limits are calculated using a Bayesian approach, with a 
flat prior distribution for the signal cross section. The 
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TABLE I. Leptoquark detection efficiencies (witli statisti- 
cal and systematic uncertainties) and 95% CL. cross section 
limits for leptoquarks in the /i/i+jets cliannel and for the com- 
bination of all decay channels a± (5 — i. Cross sections for 
Slq (NLO) and Vlq (LO) pair production are also shown. 
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FIG. 3. 95% C.L. limits on pair production cross sections. 
Results are shown for the ^/i-|-jets channel (fjjp+jots) for 
/3 = 1, |, and for all combined searches (o-j-onjijinod) si P ~ \- 



statistical and systematic uncertainties on the efficien- 
cies, tlie integrated luminosity (5%), and tlie background 
estimate are included in the calculation assuming Gaus- 
sian prior distributions. It should be noted that the cross 
section limits for the ///i+jcts channel are independent of 
/3, which enters only when comparing experimental limits 
with theory. A particular (3 is given for the combined re- 
sult since that value determines the relative contribution 
of each channel to the total cross section. 

The dominant (10%) systematic uncertainty in the 
efficiencies is due to uncertainty in the simulation. 
In addition, there are approximately equal uncertain- 
ties in the jet energy scale [E2i and the trigger effi- 
ciency/spectrometer resolution for high-pT muons (6.6% 
and 6.4% respectively). 

Figure 3 shows the limits on the pair production cross 
sections for scalar and vector leptoquarks obtained from 
this search, corrected for the branching ratio (BR = 0^ 
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FIG. 4. The regions in the /3 — rriLQ plane excluded by 
combining the results of the /^/i-|-jets, /xi'-l-jets, and w+]ets 
searches. The area to the left of each curve is excluded for 
that type of coupling, at the 95% confidence level. 
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TABLE II. Combined 95% C.L. lower mass limits for sec- 
ond generation leptoquarks. 

for /i^-|-jets). The results are given for /3 = 1 and ^. 
The lower mass limits at the 95% confidence level ob- 
tained from comparing the cross section limits with the 
theory cross sections at /Lt = 2ras^Q for the /x/i-|-jets de- 
cay channel at /3 = 1 (1/2) are: 200 (145) GeV/c^, 270 
(225) GeV/c2 and 325 (280) GeV/c^ for scalar, MV, and 
YM vector couplings, respectively. 

The results from the /i/i -|-jets (BR — 0^) search are 
combined with results from previous second generation 
leptoquark searches in the /ii^+jets (BR = 2/3(1 — /?)) q] 
and i/z^-|-jets (BR == (1 — PY) channels. Limits on the 
combined cross section (BR = 1) are listed in Table I, 
for (3 = 1/2. These limits are also shown in Fig. 3, and 
the lower mass limits obtained are: 180 GeV/c^ (-^lq), 
260 GeV/c2 (MV), and 310 GeV/c^ (YM), aU at the 
95% confidence level. Mass limits calculated from the 
combination of channels as a function of [3 are shown in 
Fig. 4 and summarized in Table II. 

In conclusion, a search has been performed for second 
generation leptoquark pairs decaying via LQ — > /xg using 
94 ± 5 pb^^ of data. No evidence is found for a signal, 
and limits are set at the 95% confidence level on the mass 
of second generation leptoquarks. By combining these re- 
sults with those from previous studies comprehensive lim- 
its on second generation leptoquarks are obtained. These 
are shown as exclusion contours constraining the possible 
values of (3 and tolq by coupling. 
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